Do politicians face consequences for disagreeing with their party? Do consumers prefer familiar or unconventional cultural items? When hiring, do companies favor typical or atypical career backgrounds? Are research articles that adhere to field norms more impactful? Are patent applications for innovations fitting a known technology class approved more often? One way to answer these questions empirically involves measuring the semantic similarity between objects and concepts (or categories). Research in social and behavioral science refers to such an assessment as “perceived typicality.” Typicality is the degree to which an object is perceived as representative or prototypical of a given concept.Accurately measuring typicality has proved challenging. In our paper just published in PNAS, we show how to do this with ChatGPT. To our surprise, we realized it was enough to “just ask” ChatGPT for a typicality rating to obtain state-of the-art performance (in terms of match to human judgments). What is amazing about these results is that performance is better than that obtained with models (even based on recent LLMs such as BERT or GPT-3) trained on hundreds of thousands of text documents. Assembling data to train an algorithm to process text data is very costly since it requires hiring human judges (experts, or crowd workers) to read many text documents and respond to questions about these. This easily costs several thousand euros for one data set. Our results show that using ChatGPT dramatically reduces the cost of performing text analysis for social science research. With Balazs Kovacs, Michael Hannan, and Guillem Pros Rius. We thank the European Research Council (ERC), the Ministry of Science and Innovation of Spain, the ICREA - Institució Catalana de Recerca i Estudis Avançats, the Barcelona School of Economics, Universitat Pompeu Fabra, Yale School of Management and Stanford University Graduate School of Business for supporting our research.
首页 >
Uncovering the semantics of concepts using GPT > Ulrik Nash on LinkedIn: Uncovering the semantics of concepts using GPT